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Abstract 

The evaluation of incomplete satisfiability solvers depends critically on the availability of hard satis- 
fiable instances. A plausible source of such instances consists of random fc-SAT formulas whose clauses 
are chosen uniformly from among all clauses satisfying some randomly chosen truth assignment A. Un¬ 
fortunately, instances generated in this manner tend to be relatively easy and can be solved efficiently 
by practical heuristics. Roughly speaking, as the formula’s density increases, for a number of different 
algorithms, A acts as a stronger and stronger attractor. Motivated by recent results on the geometry of 
the space of satisfying truth assignments of random fc-SAT and NAE-fe-SAT formulas, we introduce a 
simple twist on this basic model, which appears to dramatically increase its hardness. Namely, in addi¬ 
tion to forbidding the clauses violated by the hidden assignment A, we also forbid the clauses violated by 
its complement, so that both A and A are satisfying. It appears that under this “symmetrization” the 
effects of the two attractors largely cancel out, making it much harder for algorithms to find any truth 
assignment. We give theoretical and experimental evidence supporting this assertion. 


1 Introduction 

Recent years have witnessed the rapid development and application of search methods for constraint satis¬ 
faction and Boolean satisfiability. An important factor in the success of these algorithms is the availability 
of good sets of benchmark problems to evaluate and fine-tune them. There are two main sources of such 
problems: the real world, and random instance generators. Real-world problems are arguably the best 
benchmark, but unfortunately are often in short supply. Moreover, using real-world problems carries the 
risk of tuning algorithms toward the specific application domains for which good benchmarks are available. 
In that sense, random instance generators are a good additional source, with the advantage of controllable 
characteristics, such as size and expected hardness. 

Hard random instances have led to the development of new stochastic search methods such as WalkSAT 
m and the breakout procedure m, and have been used in detailed comparisons of local search methods 
for graph coloring and related graph problems ini The results of various competitions for CSP and SAT 
algorithms show a fairly direct correlation between the performance on real-world benchmarks and on hard 
random instances P | [T ^ [T7|. Nevertheless, a key limitation of current problem generators concerns their 
use in evaluating incomplete satisfiability solvers such as those based on local search methods. 

When an incomplete algorithm does not find a solution, it can be difficult to determine whether this is 
because the instance is in fact unsatisfiable, or simply because the algorithm failed to find the satisfying 
assignment. The standard way of dealing with this problem is to use a complete search method to filter out 
the unsatisfiable cases. However, this greatly limits the size and difficulty of problem instances that can be 
considered. Ideally, one would use problem generators that generate satisfiable instances only. One relatively 
recent source of such problems is the quasigroup completion problem isniEiiini. However, a generator for 
random hard satisfiable instances of 3-SAT, say, has remained elusive. 

Perhaps the most natural candidate for generating random hard satisfiable 3-SAT formulas is the follow¬ 
ing. Pick a random truth assignment A, and then generate a formula with n variables and rn random clauses, 
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rejecting any clause that is violated by A. In particular, we might hope that if we work close to the satis¬ 
fiability threshold region r 4.25, where the hardest random 3-SAT problems seem to be nni 1121123, this 
would generate hard satisfiable instances. Unfortunately, this generator is highly biased towards formulas 
with many assignments clustered around A. When given to local search methods such as WalkSAT, the re¬ 
sulting formulas turn out to be much easier than formulas of comparable size obtained by filtering satisfiable 
instances from a 3-SAT generator. More sophisticated versions of this “hidden assignment” scheme 13 EDI 
improve matters somewhat but still lead to easily solvable formulas. 

In this paper we introduce a new generator of random satisfiable problems. The idea is simple: we pick a 
random 3-SAT formula that has a “hidden” complementary pair of satisfying assignments, A and A, by 
rejecting clauses that are violated by either A or A. We call these “2-hidden” formulas. Our motivation comes 
from recent work 013 which showed that moving from random fc-SAT to random NAE-fc-SAT (in which 
every clause in the formula must have at least one true and at least one false literal) tremendously reduces 
the correlation between solutions. That is, whereas in random fc-SAT, satisfying assignments tend to form 
clumps, in random NAE-fc-SAT the solutions appear to be scattered throughout {0,1}" in a rather uniform 
“mist”, even for densities extremely close to the threshold. An intuitive explanation for this phenomenon is 
that since the complement of every NAE-assignment is also an NAE-assignment, the attractions of solution 
pairs largely “cancel out.” In this paper we exploit this phenomenon to impose a similar symmetry on the 
hidden assignments A and A, so that their attractions cancel out, making it hard for a wide variety of 
algorithms to “feel” either one. 

A particularly nice feature of our generator is that it is based on an extremely simple probabilistic 
procedure, in sharp contrast with 3-SAT generators based on, say, cryptographic ideas m In particular, 
our generator is readily amenable to all the mathematical tools that have been developed for the rigorous 
study of random A:-SAT formulas. Here we make two first steps in that direction. In Section 13 via a first 
moment calculation we study the distribution of the number of solutions as a function of their distance from 
the hidden assignments. In Section 0 on the other hand, we use the technique of differential equations to 
analyze the performance of the Unit Clause (UC) heuristic on our formulas. 

Naturally, mathematical simplicity would not be worth much if the formulas produced by our generator 
were easily solvable. In Section0 we compare experimentally the hardness of “2-hidden” formulas with that 
of “l-hidden” and “0-hidden” formulas. That is, we compare our formulas with random 3-SAT formulas with 
one hidden assignment and with standard random 3-SAT formulas with no hidden assignment. We examine 
four leading algorithms: two complete solvers, zChaff and Satz, and two incomplete ones, WalkSAT and the 
recently introduced Survey Propagation (SP). 

For all these algorithms, we find that our formulas are much harder than l-hidden formulas and, more 
importantly, about as hard as 0-hidden formulas, of the same size and density. 


2 A picture of the space of solutions 

In this section we compare l-hidden and 2-hidden formulas with respect to the expected number of solutions 
at a given distance from the hidden assignment(s). 

2.1 l-hidden formulas 

Let X be the number of satisfying truth assignments in a random fc-SAT formula with n variables and 
m = rn clauses chosen uniformly and independently among all /c-clauses with at least one positive literal, 
i.e., l-hidden formulas where we hide the all-ones truth assignment. To calculate the expectation E[A], it 
is helpful to parametrize truth assignments according to their overlap with the hidden assignment, i.e., the 
fraction a of variables on which they agree with A, which in this case is the fraction of variables that are set 
to one. Then, linearity of expectation gives o, clause independence gives @, selecting the literals in each 
clause uniformly and independently gives @, and, finally, writing z = an and using Stirling’s approximation 
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for the factorial gives 0 : 
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where 

1 f 1 k 

fk,r{a) - yl- 2k _ij 

From this calculation we see that E[X] is dominated by the contribution of the truth assignments that 
maximize fk,rio:) (since we raise fk,r to the nth power all other contributions vanish). Now, we readily 
see that / is the product of an “entropic” factor l/(a“(l — a)^~°‘) which is symmetric around a = 1/2, 
and a “correlation” factor which is strictly increasing in a. As a result, it is always maximized for some 
a > 1/2. This means that the dominant contribution to E[X] comes from truth assignments that agree with 
the hidden assignment on more that half the variables. That is, the set of solutions is dominated by truth 
assignments that can “feel” the hidden assignments. Moreover, as r increases this phenomenon becomes 
more and more acute (see Figure ^ below). 


2.2 2-hidden formulas 


Now let X be the number of satisfying truth assignments in a random fc-SAT formula with n variables and 
m = rn clauses chosen uniformly among all /c-clauses that have at least one positive and at least one negative 
literal, i.e., 2-hidden formulas where we hide the all-ones assignment and its complement. To compute E)^] 
we proceed as above, except that now © is replaced by 



Carrying through the ensuing changes we find that now 


E[X] = poly(n) X max ^(a)]” 

qG[0.1] 


where 


9k,r{a) 


1 / 1 - (1 - a)'= 

a“(l - a)i-“ V 2 ^= - 2 


r 


This time, both the entropic factor and the correlation factor comprising g are symmetric functions of a, so 
gk,r is symmetric around a = 1/2 (unlike fk,r)- Indeed, one can prove that for all r up to extremely elose to 
the random fc-SAT threshold r^, the function g^^r has its global maximum at a = 1/2. In other words, for all 
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such r, the dominant contribution to E[X] comes from truth assignments at distance n/2 from the hidden 
assignments, i.e., the hidden assignments are “not felt.” More precisely, there exists a sequence Cfc —> 0 such 
that gk^r has a unique global maximum at a = 1/2, for all 

r < 2^^ ln2 - - 1 - e/c . (5) 

Contrast this with the fact (implicit in [201) that for 

r>2'=ln2-i^-i , (6) 

a random fc-SAT formula with n variables and m = rn clauses is unsatisfiable with probability 1 — o(l). 
Moreover, the convergence of the sequence efc —> 0 is rapid, as can be seen from the concrete values in tabled 


k 

3 

4 

5 

7 

10 

20 

Eq. © 

7/2 

35/4 

20.38 

87.23 

708.40 

726816.15 

Eq. ® 

4.67 

10.23 

21.33 

87.88 

708.94 

726816.66 


Table 1: The convergence (in k) to the asymptotic gap of 1/2 is rapid 

Below we plot fk,r and for k = 5 and r = 16,18, 20, 22, 24 (from top to bottom). We see that in the 
case of 1-hidden formulas, i.e., fk,r, the maximum always occurs to the right of a = 1/2. Moreover, observe 
that for r = 22,24, i.e., after we cross the 5-SAT threshold (which occurs at r « 21) we have a dramatic 
shift in the location of the maximum and, thus, in the extent of the bias: as one would expect, the only 
remaining satisfying assignments above the threshold are those extremely close to the hidden assignment. 

In the case of 2-hidden formulas, on the other hand, we see that for r = 16,18, 20 the global maximum 
occurs at a = 1/2 (from the table above we know that the critical r for fc = 5 is 20.38). For r = 20, we 
also have two local maxima, near a = 0,1, but since gk,r is raised to the nth power, these are exponentially 
suppressed. Naturally, for r above the threshold, i.e., r = 22, 24, these local maxima become global, signifying 
that indeed the only remaining truth assignments are those extremely close to one of the two hidden ones. 

Intuitively, we expect that because g is flat at a = 1/2 where random truth assignments are concentrated, 
for 2-hidden formulas local search algorithms like WalkSAT will essentially perform a random walk until they 
are lucky enough to get close to one of the two hidden assignments. Thus we expect WalkSAT to take about 
as long on 2-hidden formulas as it does on 0-hidden ones. For 1-hidden formulas, in contrast, we expect 
the nonzero gradient of / at a = 1/2 to provide a strong “hint” to WalkSAT that it should move towards 
the hidden assignment, and that therefore 1-hidden formulas will be much easier for it to solve. We will see 
below that our experimental results bear out these intuitions perfectly. 

3 The Unit Clause heuristic and DPLL algorithms 

Consider the following linear-time heuristic, called Unit Clause (UC), which permanently sets one variable in 
each step as follows: pick a random literal and satisfy it; repeatedly satisfy any 1-clauses present. In (Sj, Chao 
and Franco showed that UC succeeds with constant probability on random 3-SAT formulas with r < 8/3, 
and fails with high probability, i.e., with probability I — o(I) as n oo, for r > 8/3. One can think of UC 
as the first branch of the simplest possible DPLL algorithm S: set variables in a random order, each time 
choosing randomly which branch to take first. The result of |2j then shows that, with constant probability, 
S solves random 3-SAT formulas with r < 8/3 with no backtracking at all. 

Conversely, calculations from statistical physics unim suggest that with high probability S takes expo¬ 
nential time for all r > 8/3. That is, around r = 8/3 the running time of S goes from linear to exponential, 
with no intermediate regime. In |2], it was proved that S takes exponential time for r > 3.81, which is 
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1-hidden formulas 2-hidden formulas 

Figure 1: The nth root of the expected number of solutions fk,r and gk,r for 1-hidden and 2-hidden formulas 
respectively, as a function of the overlap fraction a = zin with the hidden assignment. Here k = 5 and 
r = 16,18, 20, 22, 24 from top to bottom. 


already well below the conjectured satisfiability threshold r « 4.2. Moreover, the results in [2] imply that if 
the “tricritical point” of (2 -|-p)-SAT is r = 2/5, one can replace 3.81 with 8/3. 

In this section we analyze the performance of UC on 1-hidden and 2-hidden formulas. Specifically, we 
show that UC fails for 2-hidden formulas at precisely the same density as for 0-hidden ones. Based on this, we 
conjecture that the running time of S', and other simple DPLL algorithms, becomes exponential for 2-hidden 
formulas at the same density as for 0-hidden ones. 

To analyze UC on random 1-hidden and 2-hidden formulas we actually analyze UC on arbitrary initial 
distributions of 3-clauses, i.e., where for each 0 < j < 3 we specify the initial number of 3-clauses with j 
positive literals and 3 — j negative ones. We use the method of differential equations; see ^ for a review. 
To simplify notation, we assume that A is the all-ones assignment, so that 1-hidden formulas forbid clauses 
where all literals are negative, while 2-hidden formulas forbid all-negative and all-positive clauses. 

A round of UC consists of a free step, in which we satisfy a random literal, and the ensuing chain of 
unit-clause propagations. For 0 < i < 3 and 0 < j < i, let Sij = Sijn be the number of clauses of length i 
with j positive literals and i — j negative ones. We will also refer to the total density of clauses of size i as 
Si = Let X = xn he the number of variables set so far. Our goal is to write the expected change 

in these variables in a given round as a function of their values at the beginning of the round. Note that at 
the beginning of each round S'i,o = = 0 by definition, so the “state space” of our analysis will consist of 

the variables Sij for i >2. 

It is convenient to define two new quantities, mx and mp, which are the expected number of variables 
set True and False in a round. We will calculate these below. Then, in terms of tot, wf, we have 


E[A^3.,] 


E[A^2.,] 

E[AX] 


— (tot + rap) 


1 — X 


— (rriT + rap) 


2s 


2.1 


1 — X 


• m F -;-h rriT • 


1 — X 


1 — X 


— {tut + rnp) . 


(7) 

( 8 ) 


To see this, note that a variable appears positively in a clause of type i,j with probability j/{n — X), and 
negatively with probability {i — j)/{n — X). Thus, the negative terms in (71) and 0 correspond to clauses 
being “hit” by the variables set, while the positive term is the “flow” of 3-clauses to 2-clauses. 

To calculate mp and mp, we consider the process by which unit clauses are created during a round. We 
can model this with a two-type branching process, which we analyze as in j^]. Since the free step gives the 
chosen variable a random value, we can think of it as creating a unit clause, which is positive or negative 
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with equal probability. Thus the initial expected population of unit clauses can be represented by a vector 


f 1/2 

= [ 1/2 

where the first and second components count the negative and positive unit clauses respectively. Moreover, 
at time X = xn, a unit clause procreates according to the matrix 


M = 


1 


1 — X 


S2,l 2 s2,0 
2S2,2 S2,1 


In other words, satisfying a negative unit clause creates, in expectation, Mi_i = S2,i/(1 — x) negative unit 
clauses and M 2 ,i = 2 s 2 . 2 /(l — x) positive unit clauses, and similarly for satisfying a positive unit clause. 

Thus, as long as the largest eigenvalue Ai of M is less than 1, the expected number of variables set true 
or false during the round is given by 


/ mp 
V Tot 


(/ + M + + ...). po = (/ - M)-^ ■ po 


where I is the identity matrix. Moreover, as long as Ai < 1 throughout the algorithm, i.e., as long as the 
branching process is subcritical for all x, UC succeeds with constant probability. On the other hand, if Ai 
ever exceeds 1, then the branching process becomes supercritical, with high probability the unit clauses 
proliferate and the algorithm fails. Note that 


Ai — 


S2,l + 2.^32,0 S2,2 

I — X 


(9) 


Now let us rescale 0 to give a system of differential equations for the Sij. Wormald’s Theorem m implies 
that w.h.p. the random variables Sij(xn) will be within o(n) of Sij(x) ■ n for all x: 


ds3j _ 3 s3j 

dx 1 — X 

dg2j ^ 2 s2j ^_ 

dx 1 — X rriT + mp 


( 10 ) 


(j + l)s3j+i , rriT (3 - j)s3j 


1 — X 


rriT + mp 1 — x 


Now, suppose our initial distribution of 3-clauses is symmetric, i.e., S 3 p( 0 ) = 33 , 3 ( 0 ) and 33 , 1 ( 0 ) = 33 , 2 ( 0 ). 
It is easy to see from m that in that case, both the 3-clauses and the 2-clauses are symmetric at all times, 
i.e., Sij = Si^i-j and mp = mp- In that case 32,1 -I-2^32, 0S2,2 = 32, so the criterion for subcriticality becomes 


Ai = 


52 

1 — X 


< 1 


Moreover, since the system m is now symmetric with respect to j, summing over j gives the differential 
equations 


333 
1 — X 

232 333 

1 — X 2(1 — x) 

which are precisely the differential equations for UC on 0-hidden formulas, i.e., random instances of 3-SAT. 

Since 2-hidden formulas correspond to symmetric initial conditions, we have thus shown that UC succeeds 
on them with constant probability if and only if r < 8/3, i.e., that UC fails on these formulas at exactly the 
same density for which it fails on random 3-SAT instances. (In contrast, integrating 1101) with the initial 
conditions corresponding to 1-hidden formulas shows that UC succeeds for them at a slightly higher density, 
up to r < 2.679.) 


d33 

dx 

d32 

dx 
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Of course, UC can easily be improved by making the free step more intelligent: for instance, choosing the 
variable according the number of its occurrences in the formula, and using the majority of these occurrences 
to decide its truth value. The best known heuristic of this type [Eiini succeeds with constant probability for 
r < 3.52. However, we believe that much of the progress that has been made in analyzing the performance of 
such algorithms can be “pushed through” to 2-hidden formulas. Specifically, nearly all algorithms analyzed 
so far have the property that given as input a symmetric initial distribution of 3-clauses, e.g. random 3-SAT, 
their residual formulas consist of symmetric mixes of 2- and 3-clauses. As a result, we conjecture that the 
above methods can be used to show that such algorithms act on 2-hidden formulas exactly as they do on 
0-hidden ones, failing w.h.p. at the same density. 

More generally, call a DPLL algorithm myopic if its splitting rule consists of choosing a random clause 
of a given size, based on the current distribution of clause sizes, and deciding how to satisfy it based on the 
number of occurrences of its variables in other clauses. For a given myopic algorithm A, let ta be the density 
below which A succeeds without any backtracking with constant probability. The results of imply the 
following statement: if the tricritical point for random (2 -|-p)-SAT is pc = 2/5 then every myopic algorithm 
A takes exponential time for r > ta- Thus, not only UC, but in fact a very large class of natural DPLL 
algorithms, would go from linear time for r < r a to exponential time for r > r a- The fact that the linear¬ 
time heuristics corresponding to the first branch of A act on 2-hidden formulas just as they do on 0-hidden 
ones suggests that, for a wide variety of DPLL algorithms, 2-hidden formulas become exponentially hard at 
the same density as 0-hidden ones. Proving this, or indeed proving that 2-hidden formulas take exponential 
time for r above some critical density, appears to us a very promising direction for future work. 


4 Experimental results 

In this section we report experimental results on our 2-hidden formulas, and compare them to 1-hidden 
and 0-hidden ones. We use two leading complete solvers, zChaff and Satz, and two leading incomplete 
solvers, WalkSAT and the new Survey Propagation algorithm SP. In an attempt to avoid the numerous 
spurious features present in “too-small” random instances, i.e., in non-asymptotic behavior, we restricted 
our attention to experiments where n > 1000. This meant that zChaff and Satz could only be examined 
at densities significantly above the satisfiability threshold, as neither algorithm could practically solve either 
0-hidden or 2-hidden formulas with n ^ 1000 variables close to the threshold. For WalkSAT and SP, on 
the other hand, we can easily run experiments in the hardest range (around the satisfiability threshold) for 
n ~ 10^. 

4.1 zChaff and Satz 

In order to do experiments with n > 1000 with zChaff and Satz, we focused on the regime where r is 
relatively large, 20 < r < 60. As stated above, for r near the satisfiability threshold, 0-hidden and 2-hidden 
random formulas with n ^ 1000 variables seem completely out of the reach of either algorithm. While 
formulas in this overconstrained regime are still challenging, the presence of many forced steps allows both 
solvers to completely explore the space fairly quickly. 

We obtained zChaff from the Princeton web site [^T]. The left part of Figure |21 shows its performance 
on random formulas of all three types (with n = 1000 for 20 < r < 40 and n = 3000 for 40 < n < 60). We 
see that the number of decisions for all three types of problems decreases rapidly as r increases, consistent 
with earlier findings for complete solvers on random 3-SAT formulas. 

Figure El shows that zChaff finds 2-hidden formulas almost as difficult as 0-hidden ones, which for this 
range of r are unsatisfiable with overwhelming probability. On the other hand, the 1-hidden formulas are 
much easier, with a number of branchings between 2 and 5 orders of magnitude smaller. It appears that 
while zChaff’s smarts allow it to quickly “zero in” on a single hidden assignment, the attractions exerted 
by a complementary pair of assignments do indeed cancel out, making 2-hidden formulas almost as hard as 
unsatisfiable ones. That is, the algorithm eventually “stumbles” upon one of the two hidden assignments 
after a search that is nearly as exhaustive as for the unsatisfiable random 3-SAT formulas of the same density. 
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Figure 2: The median number of branchings made by zChaf f and Satz on random instances with 0, 1, and 
2 hidden assignments (on a logj^g scale). For zChaff we use n = 1000 for r = 20,30,40 and n = 3000 for 
r = 40, 50, 60, and for Satz we use n = 3000 throughout. Each point is the median of 25 trials. The 2-hidden 
formulas are almost as hard for both algorithms as the 0-hidden ones, while the 1-hidden formulas are much 
easier for zChaff. 


We obtained Satz from the SATLIB web site m The right part of Figure |2] shows experiments on 
random formulas of all three types with n = 3000. As for zChaff, the 2-hidden formulas are almost as hard 
for Satz as the 0-hidden formulas are. On the other hand, the 1-hidden formulas are too. Indeed, Figure El 
shows that the median number of branches for all three types of formulas is within a multiplicative constant. 

The reason for this is simple: while Satz makes intelligent decisions about which variable to branch on, 
it tries these branches in a fixed order, attempting first to set each variable false m Therefore, a single 
hidden assignment will appear at a uniformly random leaf in Satz’s search tree. In the 2-hidden case, since 
the two hidden assignments are complementary, one will appear in a random position and the other one in 
the symmetric position with respect to the search tree. Naturally, trying branches in a fixed order is a good 
idea when the true goal is to prove that a formula is unsatisfiable, e.g. in hardware verification. However, 
we expect that if Satz were modified to, say, use the majority heuristic to choose a variable’s first value, its 
performance on the three types of problems would be similar to zChaff’s. 

4.2 SP 

SP is an incomplete solver recently introduced by Mezard and Zecchina m based on a generalization of belief 
propagation the authors call survey propagation. It is inspired by the physical notion of “replica symmetry 
breaking” and the observation that for 3.9 < r < 4.25, random 3-SAT formulas appear to be satisfiable, but 
their satisfying assignments appear to be organized into clumps. 

In FigureElwe compare SP’s performance on the three types of problems near the satisfiability threshold. 
For n = 10^ SP solves 2-hidden formulas at densities somewhat above the threshold, up to r « 4.8, while it 
solves the I-hidden formulas at still higher densities, up to r 5.6. 

Presumably the 1-hidden formulas are easier for SP since the “messages” from clauses to variables, like the 
majority heuristic, tend to push the algorithm towards a hidden assignment. Having two hidden assignments 
appears to cancel these messages out to some extent, causing SP to fail at a lower density. However, this 
argument does not explain why the SP-threshold for 2-hidden formulas should be higher than the satisfiability 
threshold; nor does it explain why SP does not solve 1-hidden formulas for arbitrarily large r. Indeed, we 
















r 


Figure 3: The fraction of problems successfully solved by SP as a function of density, with n = 10^ and 30 
trials for each value of r. The threshold for solving 2-hidden formulas is somewhat higher than for 0-hidden 
ones, and for 1-hidden formulas it is higher still. 


find this latter result surprising, since as r increases the majority of clauses should point more and more 
consistently towards the hidden assignment in the 1-hidden case. 

We note that we also performed the above experiments with n = 2 x 10^ and with 5000 iterations, instead 
of the default 1000, for SP’s convergence procedure. The thresholds of Figure |21 for 1-hidden and 2-hidden 
formulas appeared to be stable under both these changes, suggesting that they are not merely artifacts of 
our particular experiments. We propose investigating these thresholds as a direction for further work. 

4.3 WalkSAT 

We conclude with a local search algorithm, WalkSAT. Unlike the complete solvers, WalkSAT can solve problems 
with n = 10^ fairly close to the threshold. We performed experiments both with a random initial state, and 
with a biased initial state where the algorithm starts with 75% agreement with one of the hidden assignments 
(note that this is exponentially unlikely). In both cases, we performed trials of 10® flips for each formula, 
without random restarts, where each step does a random or greedy flip with equal probability. Since random 
initial states w.h.p. have roughly 50% agreement with both hidden assignments, we expect their attractions 
to cancel out so that WalkSAT will have difficulty finding either of them. On the other hand, if we begin with 
a biased initial state, then the attraction from the nearby assignment will be much stronger than the other 
one; this situation is similar to a 1-hidden formula, and we expect WalkSAT to find it easily. Indeed our data 
confirms these expectations. 

In the first part of Figure 01 we measure WalkSAT’s performance on the three types of problems with 
n = 10^ and r ranging from 3.7 to 5.5, and compare them with 0-hidden formulas for r ranging from 3.7 
up to 4.1, just below the threshold where they become unsatisfiable. We see that, below the threshold, the 
2-hidden formulas are just as hard as the 0-hidden ones when WalkSAT sets its initial state randomly; indeed, 
their running times coincide to within the resolution of the figure! They both become hardest when r « 4.2, 
where 10® flips no longer suffice to solve them. 

On the other hand, the 1-hidden formulas are much easier than the 2-hidden ones, and their running 
time peaks around r = 5.2. Finally, the 2-hidden formulas are much easier to solve when we start with a 
biased initial state, in which case the running time is closer to that of 1-hidden formulas. 

In the second part of Figure 01 we compare the three types of formulas at a density very close to the 
threshold, r = 4.25, and measure their running times as a function of n. The data suggests that 2-hidden 
formulas with random initial states are much harder than 1-hidden ones, while 2-hidden formulas with biased 
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Figure 4: Left, the median number of flips needed by WalkSAT for formulas of all three types below and 
above the threshold, with n = 10^. Below the threshold, 2-hidden formulas are just as hard as 0-hidden ones 
(they coincide to within the resolution of the figure) and their running time increases steeply as we approach 
the threshold. Both above and below the threshold, 2-hidden formulas are much harder than 1-hidden ones, 
unless the algorithm starts with a (exponentially lucky) biased initial state. Right, the median number of 
flips needed by WalkSAT to solve the three types of formulas at r = 4.25 as a function of n. Here n ranges 
from 100 to 2000. While the median running time for all three is polynomial, the 2-hidden problems are 
much harder than the 1-hidden ones unless the have a biased initial state, with a running time that scales 
similarly to 0-hidden problems, i.e., random 3-SAT. 


initial states have running times within a constant of that of 1-hidden formulas. (Note that, consistent with 
experiments of |S], the median running time of all three types of problems is polynomial in n.) 

Based on this, we conjecture the following. Our 2-hidden formulas are just as hard for WalkSAT as 0- 
hidden ones up to the satisfiability threshold, and they are hardest at or near the threshold. Moreover, they 
are much harder than 1-hidden formulas, unless WalkSAT is lucky enough to have an initial state biased 
towards one of the hidden assignments. 

5 Conclusions 

We have introduced an extremely simple new generator of random satisfiable 3-SAT instances which is 
amenable to all the mathematical tools developed for the rigorous study of random 3-SAT instances. Exper¬ 
imentally, our generator appears to produce instances that are as hard as random 3-SAT instances, in sharp 
contrast to instances with a single hidden assignment. This hardness appears quite robust; our experiments 
have demonstrated it both above and below the satisfiability threshold, and for algorithms that use very 
different strategies, i.e., DPLL solvers (zChaff and Satz), local search algorithms (WalkSAT), and survey 
propagation (SP). 

We believe that random 2-hidden instances could make excellent satisfiable benchmarks, especially just 
around the satisfiability threshold, say at r = 4.25 where they appear to be the hardest for WalkSAT (although 
beating SP requires somewhat higher densities). 

Several aspects of our experiments suggest exciting directions for further work, including: 

1. Proving that the expected running time of natural Davis-Putnam algorithms on 2-hidden formulas is 
exponential in n for r above some critical density. 

2. Explaining the different threshold behaviors of SP on 1-hidden and 2-hidden formulas. 


10 















3. Understanding how long WalkSAT takes at the midpoint between the two hidden assignments, before 
it becomes sufficiently unbalanced to converge to one of them. 

4. Studying random 2-hidden formulas in the dense case where there are uj{n) clauses. 
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